Skip to content

[lldb] Clear thread name container before writing UTF8 bytes #134150

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 2, 2025

Conversation

z2oh
Copy link
Contributor

@z2oh z2oh commented Apr 2, 2025

llvm::convertUTF16ToUTF8String opens with an assertion that the output container is empty:

bool convertUTF16ToUTF8String(ArrayRef<char> SrcBytes, std::string &Out) {
assert(Out.empty());

It's not clear to me why this function requires the output container to be empty instead of just overwriting it, but the callsite in TargetThreadWindows::GetName may reuse the container without clearing it out first, resulting in an assertion failure:

 # Child-SP          RetAddr               Call Site
00 000000d2`44b8ea48 00007ff8`beefc12e     ntdll!NtTerminateProcess+0x14
01 000000d2`44b8ea50 00007ff8`bcf518ab     ntdll!RtlExitUserProcess+0x11e
02 000000d2`44b8ea80 00007ff8`bc0e0143     KERNEL32!ExitProcessImplementation+0xb
03 000000d2`44b8eab0 00007ff8`bc0e4c49     ucrtbase!common_exit+0xc7
04 000000d2`44b8eb10 00007ff8`bc102ae6     ucrtbase!abort+0x69
05 000000d2`44b8eb40 00007ff8`bc102cc1     ucrtbase!common_assert_to_stderr<wchar_t>+0x6e
06 000000d2`44b8eb80 00007fff`b8e27a80     ucrtbase!wassert+0x71
07 000000d2`44b8ebb0 00007fff`b8b821e1     liblldb!llvm::convertUTF16ToUTF8String+0x30 [D:\r\_work\swift-build\swift-build\SourceCache\llvm-project\llvm\lib\Support\ConvertUTFWrapper.cpp @ 88] 
08 000000d2`44b8ec30 00007fff`b83e9aa2     liblldb!lldb_private::TargetThreadWindows::GetName+0x1b1 [D:\r\_work\swift-build\swift-build\SourceCache\llvm-project\lldb\source\Plugins\Process\Windows\Common\TargetThreadWindows.cpp @ 198] 
09 000000d2`44b8eca0 00007ff7`2a3c3c14     liblldb!lldb::SBThread::GetName+0x102 [D:\r\_work\swift-build\swift-build\SourceCache\llvm-project\lldb\source\API\SBThread.cpp @ 432] 
0a 000000d2`44b8ed70 00007ff7`2a3a5ac6     lldb_dap!lldb_dap::CreateThread+0x1f4 [S:\SourceCache\llvm-project\lldb\tools\lldb-dap\JSONUtils.cpp @ 877] 
0b 000000d2`44b8ef10 00007ff7`2a3b0ab5     lldb_dap!`anonymous namespace'::request_threads+0xa6 [S:\SourceCache\llvm-project\lldb\tools\lldb-dap\lldb-dap.cpp @ 3906] 
0c 000000d2`44b8f010 00007ff7`2a3b0fe8     lldb_dap!lldb_dap::DAP::HandleObject+0x1c5 [S:\SourceCache\llvm-project\lldb\tools\lldb-dap\DAP.cpp @ 796] 
0d 000000d2`44b8f130 00007ff7`2a3a8b96     lldb_dap!lldb_dap::DAP::Loop+0x78 [S:\SourceCache\llvm-project\lldb\tools\lldb-dap\DAP.cpp @ 812] 
0e 000000d2`44b8f1d0 00007ff7`2a4b5fbc     lldb_dap!main+0x1096 [S:\SourceCache\llvm-project\lldb\tools\lldb-dap\lldb-dap.cpp @ 5319] 
0f (Inline Function) --------`--------     lldb_dap!invoke_main+0x22 [D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 78] 
10 000000d2`44b8fb80 00007ff8`bcf3e8d7     lldb_dap!__scrt_common_main_seh+0x10c [D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 288] 
11 000000d2`44b8fbc0 00007ff8`beefbf6c     KERNEL32!BaseThreadInitThunk+0x17
12 000000d2`44b8fbf0 00000000`00000000     ntdll!RtlUserThreadStart+0x2c

This stack trace was captured from the lldb distributed in the Swift toolchain. The issue is easy to reproduce by resuming from a breakpoint twice in VS Code.

I've verified that clearing out the container here fixes the assertion failure.

@z2oh z2oh requested a review from JDevlieghere as a code owner April 2, 2025 20:29
@llvmbot llvmbot added the lldb label Apr 2, 2025
@llvmbot
Copy link
Member

llvmbot commented Apr 2, 2025

@llvm/pr-subscribers-lldb

Author: Jeremy Day (z2oh)

Changes

llvm::convertUTF16ToUTF8String opens with an assertion that the output container is empty:

bool convertUTF16ToUTF8String(ArrayRef<char> SrcBytes, std::string &Out) {
assert(Out.empty());

It's not clear to me why this function requires the output container to be empty instead of just overwriting it, but the callsite in TargetThreadWindows::GetName may reuse the container without clearing it out first, resulting in an assertion failure:

 # Child-SP          RetAddr               Call Site
00 000000d2`44b8ea48 00007ff8`beefc12e     ntdll!NtTerminateProcess+0x14
01 000000d2`44b8ea50 00007ff8`bcf518ab     ntdll!RtlExitUserProcess+0x11e
02 000000d2`44b8ea80 00007ff8`bc0e0143     KERNEL32!ExitProcessImplementation+0xb
03 000000d2`44b8eab0 00007ff8`bc0e4c49     ucrtbase!common_exit+0xc7
04 000000d2`44b8eb10 00007ff8`bc102ae6     ucrtbase!abort+0x69
05 000000d2`44b8eb40 00007ff8`bc102cc1     ucrtbase!common_assert_to_stderr&lt;wchar_t&gt;+0x6e
06 000000d2`44b8eb80 00007fff`b8e27a80     ucrtbase!wassert+0x71
07 000000d2`44b8ebb0 00007fff`b8b821e1     liblldb!llvm::convertUTF16ToUTF8String+0x30 [D:\r\_work\swift-build\swift-build\SourceCache\llvm-project\llvm\lib\Support\ConvertUTFWrapper.cpp @ 88] 
08 000000d2`44b8ec30 00007fff`b83e9aa2     liblldb!lldb_private::TargetThreadWindows::GetName+0x1b1 [D:\r\_work\swift-build\swift-build\SourceCache\llvm-project\lldb\source\Plugins\Process\Windows\Common\TargetThreadWindows.cpp @ 198] 
09 000000d2`44b8eca0 00007ff7`2a3c3c14     liblldb!lldb::SBThread::GetName+0x102 [D:\r\_work\swift-build\swift-build\SourceCache\llvm-project\lldb\source\API\SBThread.cpp @ 432] 
0a 000000d2`44b8ed70 00007ff7`2a3a5ac6     lldb_dap!lldb_dap::CreateThread+0x1f4 [S:\SourceCache\llvm-project\lldb\tools\lldb-dap\JSONUtils.cpp @ 877] 
0b 000000d2`44b8ef10 00007ff7`2a3b0ab5     lldb_dap!`anonymous namespace'::request_threads+0xa6 [S:\SourceCache\llvm-project\lldb\tools\lldb-dap\lldb-dap.cpp @ 3906] 
0c 000000d2`44b8f010 00007ff7`2a3b0fe8     lldb_dap!lldb_dap::DAP::HandleObject+0x1c5 [S:\SourceCache\llvm-project\lldb\tools\lldb-dap\DAP.cpp @ 796] 
0d 000000d2`44b8f130 00007ff7`2a3a8b96     lldb_dap!lldb_dap::DAP::Loop+0x78 [S:\SourceCache\llvm-project\lldb\tools\lldb-dap\DAP.cpp @ 812] 
0e 000000d2`44b8f1d0 00007ff7`2a4b5fbc     lldb_dap!main+0x1096 [S:\SourceCache\llvm-project\lldb\tools\lldb-dap\lldb-dap.cpp @ 5319] 
0f (Inline Function) --------`--------     lldb_dap!invoke_main+0x22 [D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 78] 
10 000000d2`44b8fb80 00007ff8`bcf3e8d7     lldb_dap!__scrt_common_main_seh+0x10c [D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 288] 
11 000000d2`44b8fbc0 00007ff8`beefbf6c     KERNEL32!BaseThreadInitThunk+0x17
12 000000d2`44b8fbf0 00000000`00000000     ntdll!RtlUserThreadStart+0x2c

This stack trace was captured from the lldb distributed in the Swift toolchain. The issue is easy to reproduce by resuming from a breakpoint twice in VS Code.

I've verified that clearing out the container here fixes the assertion failure.


Full diff: https://github.com/llvm/llvm-project/pull/134150.diff

1 Files Affected:

  • (modified) lldb/source/Plugins/Process/Windows/Common/TargetThreadWindows.cpp (+1)
diff --git a/lldb/source/Plugins/Process/Windows/Common/TargetThreadWindows.cpp b/lldb/source/Plugins/Process/Windows/Common/TargetThreadWindows.cpp
index a0d0f0ea0abc8..b2b66f2927644 100644
--- a/lldb/source/Plugins/Process/Windows/Common/TargetThreadWindows.cpp
+++ b/lldb/source/Plugins/Process/Windows/Common/TargetThreadWindows.cpp
@@ -192,6 +192,7 @@ const char *TargetThreadWindows::GetName() {
   if (SUCCEEDED(GetThreadDescription(
           m_host_thread.GetNativeThread().GetSystemHandle(), &pszThreadName))) {
     LLDB_LOGF(log, "GetThreadDescription: %ls", pszThreadName);
+    m_name.clear();
     llvm::convertUTF16ToUTF8String(
         llvm::ArrayRef(reinterpret_cast<char *>(pszThreadName),
                        wcslen(pszThreadName) * sizeof(wchar_t)),

@JDevlieghere JDevlieghere merged commit be3abfc into llvm:main Apr 2, 2025
12 checks passed
@JDevlieghere
Copy link
Member

@z2oh Would you mind cherry-picking this to https://github.com/swiftlang/llvm-project/tree/stable/20240723?

@ktraunmueller
Copy link

ktraunmueller commented Apr 3, 2025

Thank you!

FYI, I had created a bug report (which can now be closed) for this problem, but forgot to add that information to the Swift forum post, sorry for that.

@ktraunmueller
Copy link

ktraunmueller commented Apr 3, 2025

I was also surprised by the precondition, which looks unnecessary and, more importantly, very unexpected, to me.

And it's not mentioned in the documentation:

/**
 * Converts a stream of raw bytes assumed to be UTF16 into a UTF8 std::string.
 *
 * \param [in] SrcBytes A buffer of what is assumed to be UTF-16 encoded text.
 * \param [out] Out Converted UTF-8 is stored here on success.
 * \returns true on success
 */
bool convertUTF16ToUTF8String(ArrayRef<char> SrcBytes, std::string &Out);

Should I prepare a PR to mention the precondition in the documentation?

@z2oh
Copy link
Contributor Author

z2oh commented Apr 3, 2025

Cherrypick is up: swiftlang#10415. Thanks @JDevlieghere!

@ktraunmueller, ah I should've checked for an issue first, you had already done the investigation! I'm in favor of the documentation change you propose. I'd also be in favor of just removing the assertion, but it's very possible I'm overlooking some reason why it's there in the first place (and it has been there for 12 years now).

@ktraunmueller
Copy link

@compnerd weighed in that assertions are always there for a reason, and I would generally agree on that 100%, but this precondition just doesn't seem to make sense, so I would agree that removing it would seem preferrable in this particular case. I can't see how that would create harm.

@ktraunmueller
Copy link

Famous last words, haha

@compnerd
Copy link
Member

compnerd commented Apr 3, 2025

I think that a fully symbolicated stack trace would be very helpful. It might explain the reason for the precondition.

@z2oh
Copy link
Contributor Author

z2oh commented Apr 3, 2025

I captured one above:

 # Child-SP          RetAddr               Call Site
00 000000d2`44b8ea48 00007ff8`beefc12e     ntdll!NtTerminateProcess+0x14
01 000000d2`44b8ea50 00007ff8`bcf518ab     ntdll!RtlExitUserProcess+0x11e
02 000000d2`44b8ea80 00007ff8`bc0e0143     KERNEL32!ExitProcessImplementation+0xb
03 000000d2`44b8eab0 00007ff8`bc0e4c49     ucrtbase!common_exit+0xc7
04 000000d2`44b8eb10 00007ff8`bc102ae6     ucrtbase!abort+0x69
05 000000d2`44b8eb40 00007ff8`bc102cc1     ucrtbase!common_assert_to_stderr<wchar_t>+0x6e
06 000000d2`44b8eb80 00007fff`b8e27a80     ucrtbase!wassert+0x71
07 000000d2`44b8ebb0 00007fff`b8b821e1     liblldb!llvm::convertUTF16ToUTF8String+0x30 [D:\r\_work\swift-build\swift-build\SourceCache\llvm-project\llvm\lib\Support\ConvertUTFWrapper.cpp @ 88] 
08 000000d2`44b8ec30 00007fff`b83e9aa2     liblldb!lldb_private::TargetThreadWindows::GetName+0x1b1 [D:\r\_work\swift-build\swift-build\SourceCache\llvm-project\lldb\source\Plugins\Process\Windows\Common\TargetThreadWindows.cpp @ 198] 
09 000000d2`44b8eca0 00007ff7`2a3c3c14     liblldb!lldb::SBThread::GetName+0x102 [D:\r\_work\swift-build\swift-build\SourceCache\llvm-project\lldb\source\API\SBThread.cpp @ 432] 
0a 000000d2`44b8ed70 00007ff7`2a3a5ac6     lldb_dap!lldb_dap::CreateThread+0x1f4 [S:\SourceCache\llvm-project\lldb\tools\lldb-dap\JSONUtils.cpp @ 877] 
0b 000000d2`44b8ef10 00007ff7`2a3b0ab5     lldb_dap!`anonymous namespace'::request_threads+0xa6 [S:\SourceCache\llvm-project\lldb\tools\lldb-dap\lldb-dap.cpp @ 3906] 
0c 000000d2`44b8f010 00007ff7`2a3b0fe8     lldb_dap!lldb_dap::DAP::HandleObject+0x1c5 [S:\SourceCache\llvm-project\lldb\tools\lldb-dap\DAP.cpp @ 796] 
0d 000000d2`44b8f130 00007ff7`2a3a8b96     lldb_dap!lldb_dap::DAP::Loop+0x78 [S:\SourceCache\llvm-project\lldb\tools\lldb-dap\DAP.cpp @ 812] 
0e 000000d2`44b8f1d0 00007ff7`2a4b5fbc     lldb_dap!main+0x1096 [S:\SourceCache\llvm-project\lldb\tools\lldb-dap\lldb-dap.cpp @ 5319] 
0f (Inline Function) --------`--------     lldb_dap!invoke_main+0x22 [D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 78] 
10 000000d2`44b8fb80 00007ff8`bcf3e8d7     lldb_dap!__scrt_common_main_seh+0x10c [D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 288] 
11 000000d2`44b8fbc0 00007ff8`beefbf6c     KERNEL32!BaseThreadInitThunk+0x17
12 000000d2`44b8fbf0 00000000`00000000     ntdll!RtlUserThreadStart+0x2c

We are reusing the TargetThreadWindows objects, so the second time that TargetThreadWindows::GetName is called, m_name has already been set, which triggers the assertion (without first clearing the string that is).

convertUTF16ToUTF8String calls Out.resize so clearly expects to be able to freely mutate the string, but this is a different precondition than requiring the string be explicitly empty. The only thing I can think of is that convertUTF16ToUTF8String is somehow trying to prevent accidental reuse/overwriting of strings, but IMO it should not the responsibility of that function to care.

@compnerd
Copy link
Member

compnerd commented Apr 7, 2025

UTF-8 is a multibyte encoding, and if there is existing content in the output string, the generated result may not be a valid string, if you passed in a buffer with \xe0 and then convert the input of a, we would get something that is invalid.

JDevlieghere pushed a commit to swiftlang/llvm-project that referenced this pull request Apr 8, 2025
…4150)

`llvm::convertUTF16ToUTF8String` opens with an assertion that the output
container is empty:
https://github.com/llvm/llvm-project/blob/3bdf9a08804a5b424fd32fef3b0089f3a6db839d/llvm/lib/Support/ConvertUTFWrapper.cpp#L83-L84

It's not clear to me why this function requires the output container to
be empty instead of just overwriting it, but the callsite in
`TargetThreadWindows::GetName` may reuse the container without clearing
it out first, resulting in an assertion failure:
```
 # Child-SP          RetAddr               Call Site
00 000000d2`44b8ea48 00007ff8`beefc12e     ntdll!NtTerminateProcess+0x14
01 000000d2`44b8ea50 00007ff8`bcf518ab     ntdll!RtlExitUserProcess+0x11e
02 000000d2`44b8ea80 00007ff8`bc0e0143     KERNEL32!ExitProcessImplementation+0xb
03 000000d2`44b8eab0 00007ff8`bc0e4c49     ucrtbase!common_exit+0xc7
04 000000d2`44b8eb10 00007ff8`bc102ae6     ucrtbase!abort+0x69
05 000000d2`44b8eb40 00007ff8`bc102cc1     ucrtbase!common_assert_to_stderr<wchar_t>+0x6e
06 000000d2`44b8eb80 00007fff`b8e27a80     ucrtbase!wassert+0x71
07 000000d2`44b8ebb0 00007fff`b8b821e1     liblldb!llvm::convertUTF16ToUTF8String+0x30 [D:\r\_work\swift-build\swift-build\SourceCache\llvm-project\llvm\lib\Support\ConvertUTFWrapper.cpp @ 88]
08 000000d2`44b8ec30 00007fff`b83e9aa2     liblldb!lldb_private::TargetThreadWindows::GetName+0x1b1 [D:\r\_work\swift-build\swift-build\SourceCache\llvm-project\lldb\source\Plugins\Process\Windows\Common\TargetThreadWindows.cpp @ 198]
09 000000d2`44b8eca0 00007ff7`2a3c3c14     liblldb!lldb::SBThread::GetName+0x102 [D:\r\_work\swift-build\swift-build\SourceCache\llvm-project\lldb\source\API\SBThread.cpp @ 432]
0a 000000d2`44b8ed70 00007ff7`2a3a5ac6     lldb_dap!lldb_dap::CreateThread+0x1f4 [S:\SourceCache\llvm-project\lldb\tools\lldb-dap\JSONUtils.cpp @ 877]
0b 000000d2`44b8ef10 00007ff7`2a3b0ab5     lldb_dap!`anonymous namespace'::request_threads+0xa6 [S:\SourceCache\llvm-project\lldb\tools\lldb-dap\lldb-dap.cpp @ 3906]
0c 000000d2`44b8f010 00007ff7`2a3b0fe8     lldb_dap!lldb_dap::DAP::HandleObject+0x1c5 [S:\SourceCache\llvm-project\lldb\tools\lldb-dap\DAP.cpp @ 796]
0d 000000d2`44b8f130 00007ff7`2a3a8b96     lldb_dap!lldb_dap::DAP::Loop+0x78 [S:\SourceCache\llvm-project\lldb\tools\lldb-dap\DAP.cpp @ 812]
0e 000000d2`44b8f1d0 00007ff7`2a4b5fbc     lldb_dap!main+0x1096 [S:\SourceCache\llvm-project\lldb\tools\lldb-dap\lldb-dap.cpp @ 5319]
0f (Inline Function) --------`--------     lldb_dap!invoke_main+0x22 [D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 78]
10 000000d2`44b8fb80 00007ff8`bcf3e8d7     lldb_dap!__scrt_common_main_seh+0x10c [D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 288]
11 000000d2`44b8fbc0 00007ff8`beefbf6c     KERNEL32!BaseThreadInitThunk+0x17
12 000000d2`44b8fbf0 00000000`00000000     ntdll!RtlUserThreadStart+0x2c
```

This stack trace was captured from the lldb distributed in the Swift
toolchain. The issue is easy to reproduce by resuming from a breakpoint
twice in VS Code.

I've verified that clearing out the container here fixes the assertion
failure.

(cherry picked from commit be3abfc)
JDevlieghere added a commit to swiftlang/llvm-project that referenced this pull request Apr 8, 2025
…ead-name-container

[🍒] [lldb] Clear thread name container before writing UTF8 bytes (llvm#134150)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants